Mixed risk-neutral/minimax control of discrete-time, finite-state Markov decision processes

نویسندگان

  • Stefano P. Coraluppi
  • Steven I. Marcus
چکیده

This paper addresses the control design problem for discrete-time, nite-state Markov Decision Processes (MDPs), when both risk-neutral and minimax objectives are of interest. We introduce the mixed risk-neutral/minimax objective, and utilize results from risk-neutral and minimax control to derive an information state process and dynamic programming equations for the value function. We synthesize optimal control laws both on the nite and innnite horizon. We study the eeectiveness of both the mixed risk-neutral/minimax family and the risk-sensitive family of controllers as tools to trade oo risk-neutral and minimax objectives. We conclude that the mixed risk-neutral/minimax family is more eeective, at the cost of increased controller complexity.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Risk - Sensitive , Minimax , and Mixed Risk - Neutral / Minimax Control of Markov Decision Processes

This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, nite-state Markov Decision Processes (MDPs). We synthesize optimal policies with respect to both criteria, both for nite horizon and discounted in nite horizon problems. A generalized decision-making framework is introduced, leading to stationary risk-sensitive and minimax optimal policies on the in ...

متن کامل

Mixed Risk-Neutral/Minimax Control of Markov Decision Processes

This paper introduces a formulation of the mixed risk-neutral/minimax control problem for Markov Decision Processes (MDPs). Drawing on results from risk-neutral control and minimax control, we derive an information state process and dynamic programming equations for the value function. Furthermore, we develop a methodology to synthesize an optimal control law on the nite horizon, and a near-opt...

متن کامل

Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes

This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, nite-states Markov Decision Processes (MDPs). We synthesize optimal policies with respect to both criteria, both for nite horizon and discounted in nite horizon problem. A generalized decision-making framework is introduced, which includes as special cases a number of approaches that have been consid...

متن کامل

Risk-Sensitive and Average Optimality in Markov Decision Processes

Abstract. This contribution is devoted to the risk-sensitive optimality criteria in finite state Markov Decision Processes. At first, we rederive necessary and sufficient conditions for average optimality of (classical) risk-neutral unichain models. This approach is then extended to the risk-sensitive case, i.e., when expectation of the stream of one-stage costs (or rewards) generated by a Mark...

متن کامل

Optimal Finite-time Control of Positive Linear Discrete-time Systems

This paper considers solving optimization problem for linear discrete time systems such that closed-loop discrete-time system is positive (i.e., all of its state variables have non-negative values) and also finite-time stable. For this purpose, by considering a quadratic cost function, an optimal controller is designed such that in addition to minimizing the cost function, the positivity proper...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • IEEE Trans. Automat. Contr.

دوره 45  شماره 

صفحات  -

تاریخ انتشار 2000